Coupling Supervised and Unsupervised Techniques in Training Feed-Forward Nets

A popular approach to training feed-forward nets is to treat the problem of adaptation as a function approximation and to use curve fitting techniques. We discuss here the problems which the use of pure curve fitting techniques entail for the generalization capability and robustness of the net. These problems are in general inherently associated with the use of pure supervised learning techniques. We argue that a better approach to the training of feed-forward nets is to use adaptive techniques that combine properties of both supervised and unsupervised learning. A new formulation of the training problem is presented here. According to this formulation the net is viewed as two coupled sub-nets the first of which is trained by an unsupervised learning technique and the second by a supervised one. The same formulation gives rise to analytic expressions of the goals of the adaptation and leads to a new method for the adaptation of feed-forward nets.